This invention relates to the provision of video data over a communications medium. In particular it relates to the capture and provision of video extracts or ‘highlights’ of a recorded or live broadcast programme for compilation into a short video sequence. The provision of such highlights is an important part of many television services, particularly in sports coverage.
The extracts required to compile a sequence of “highlights” clips are typically created from within a longer video sequence using standard digital video editing tools that define the start and end points of each video clip either directly or by reference to a key event within the clip. Each clip is typically saved as an independent file which can subsequently be provided to viewers. Typically a viewer requesting a “highlights” sequence is provided with an address, or set of addresses, from which to retrieve the clips that make up the required sequence.
It is known, for example from International Patent Specification WO2004/025508 to provided an automated procedure to identify ‘highlights’ within a video sequence in order to provide the viewer with navigation aids or to simplify and reduce costs in video production and editing. The prior art typically identifies video clips automatically through analysis of changes in video or audio levels within a video sequence. More specifically, the boundaries of the ‘highlight’ video segment are typically identified using various super-histograms, frame signatures, cut detection methods, closed caption information, audio information, and so on, by analysing the visual, audio and transcript portions of the video signal. For example, based on the volume of audience cheers (U.S. Pat. No. 7,831,112) recorded in response to the performance of the athletes, or analysis of the area of the image depicting the score to detect changes (U.S. Pat. No. 7,983,442). It is also known, for example from European Patent Application EP1421780, to provide for a viewer to manually bookmark a ‘highlight’ for later access.
These approaches define each video clip by specifying specific time-codes for its start and end. This approach to defining a highlight clip is inherently inflexible, in that it only defines the length of clip, with no further information regarding the key events within that clip such as a goal scored in a football match. Similarly, the definition of a clip by start and end points necessarily determines the length of each clip, and of the resulting sequence of highlights, which dictates the time required both to deliver and to view the entire sequence.
Standard video delivery mechanisms download video sequences in segments, delivered in sequence from the start to the end of each clip. Occasions can occur when, for example because of a loss of connection, a video clip could only be partially downloaded, consequently showing the build-up to, but not including the key moment of the actual goal (or save), match point (won or saved), catch (dropped or otherwise), putt (holed or missed) etc.
Such an interruption may also cause subsequent clips in the sequence to fail to be delivered altogether. This is a particular problem as the viewer will miss the climax and final outcome of the sporting fixture if the later clips are missing.
Different viewers may also wish to view “highlights” sequences of different duration, by varying the number and/or length of individual clips in the sequence.
The absence of information on key events within the clip also impedes intelligent delivery and playback when the clip is one of a series of clips which are to be played sequentially within a highlights show or playlist.
The present invention uses a novel method to identify and specify a ‘highlight’ within a video sequence, which can then be used to improve the delivery of video highlight clips and provide enhanced video playback facilities to viewers.
The present invention provides a method of generating video clips from a sequence of video data for delivery and playback on demand, wherein the sequence is made up of discrete elements of video data, wherein each clip is identified by an event marker and a pre/post event ratio, the event marker defining one of the discrete elements in the sequence, and the pre/post event ratio determines the relative relative proportions of the duration of the clip before the element defined by the event marker and the duration of the clip after the element defined by the event marker, and wherein, in response to a request for video clip data, identified with reference to the event marker, the discrete elements of video data making up each clip are allocated to a sequence starting with the element associated with the event marker and following in an order determined by the pre/post event ratio.
The invention also provides a video service platform for generating video clips from a sequence of video data for delivery and playback on demand, comprising:                a video store for storing video sequences in the form of series of discrete elements of video data        a retrieval system for retrieving the discrete elements of video data        a client interface for receiving a request for a sequence of one or more video clips, each clip requested being defined by an event marker identifying an element of the video data,        an event ratio determination processor for identifying a pre/post event ratio associated with each clip requested, the pre/post event ratio determining the relative relative proportions of the duration of the clip before the element defined by the event marker and the duration of the clip after the element defined by the event marker,        wherein the retrieval processor is arranged to retrieve the discrete elements of video data, the order in which they are retrieved being, determined by the event markers and pre-post event ratios associated with the requested data such that the discrete elements making up each clip are allocated to a sequence starting with the element associated with the event marker and following in an order determined by the pre/post event ratio.        
Complementary to the video service platform, the invention also provides a video service platform for generating video clips from a sequence of video data for delivery and playback on demand, comprising:                a video store for storing video sequences in the form of series of discrete elements of video data        a retrieval system for retrieving the discrete elements of video data        a client interface for receiving a request for a sequence of one or more video clips, each clip requested being defined by an event marker identifying an element of the video data,        an event ratio determination processor for identifying a pre/post event ratio associated with each clip requested, the pre/post event ratio determining the relative duration of the clip before and after an element defined by the event marker,        wherein the retrieval processor is arranged to retrieve the discrete elements of video data, the order in which they are retrieved being determined by the event markers and pre-post event ratios associated with the requested data such that the discrete elements making up each clip are allocated to a sequence starting with the element associated with the event marker and following in an order determined by the pre/post event ratio.        
In the preferred arrangement the pre-post event ratio is defined by the event marker. This allows different event types to be delivered in different ways.
The invention may be used to limit the overall duration of a clip or set of clips by dropping discrete elements from the clip in reverse order of the sequence. The invention may also be used to deliver the discrete elements in accordance with their positions in the sequence, thus ensuring that in the event of a loss of communication connection the segments that have been delivered before such interruption are the most significant for creating a shortened highlights sequence.
A plurality of clips may be generated, the elements forming the clips are formed into a single sequence determined by their respective event markers and pre-post event ratios.
In order to reduce the latency in delivery caused by the elements being delivered in an order different from that in which they are to be played back, elements forming one or more clips early in the order in which the clips are to be played may be delivered at a higher compression rate than elements forming subsequent clips.
Additional clips and event markers may be added to the database during delivery of video elements to a client device, the data elements relating to the additional clip being inserted into the delivery sequence according to the relative priority orders of additional clips and those forming the original sequence, thus allowing a “catch-up” service to be updated during delivery.
A predetermined maximum clip or sequence length may be defined in terms of the time taken to deliver the video elements, thereby defining a maximum download time, which prevents overloading a delivery system with low bandwidth. The predetermined maximum may instead be defined in terms of the duration of the sequence represented by the video elements, thereby defining the time taken to view the resulting video sequence. In either case, the delivery process may be terminated if connection with the requesting device is lost, but because the elements are delivered with the event marker first, the key moment defined by the event marker is present.
A plurality of clips may be delivered, the order in which the elements forming the clips are to be delivered being determined by their respective event markers and pre-post event ratios. Thus the elements forming the sequence of clips may be prioritised for delivery in an order corresponding to both the significance of the events identified by their respective event markers, and their relative proximities to their respective event markers.
This invention utilises a mechanism that allows prioritisation of video chunks (elements) based on their relationship to the event in order to flexibly deliver and render video onto a client device with respect to restraints in bandwidth or available time
This invention uses an event time-code and knowledge of the event to define a video clip in terms of a single event time-code related to a key event within the video sequence, and a ratio. The ratio determines the relative durations of the clip before and after the instant identified by the time code. The event time-code can be further refined through the use of a user profile, and comparison to and aggregation of multiple ‘Event Markers’ provided by other viewers, as described in our co-pending application having applicant's reference A32513. In the preferred method, the receiving client device defines the duration of the clip, but in an alternative embodiment the definition and provision of clips may be undertaken by a network-based server based on knowledge of the end client, network connection and user, for example through the provision of a customised manifest file. The user device can therefore request clips from a server and composes the clips into a video sequence in accordance with the required durations of the clips.
The invention defines a video clip from a single ‘bookmark’, using metadata associated with the bookmark which enables the optimised delivery and dynamic compilation of sports video ‘highlight’ clips in bandwidth and time constrained situations. It also provides viewers with the ability to ‘bookmark’ favourite parts of a sports TV programme, enabling them to navigate to a specific event within the programme and share video clips.
The invention enables viewers to identify a number of key events within a video sequence (such as goals in a football match) using viewer-defined ‘temporal bookmarks’. These bookmarks are stored as time-codes along with associated metadata which identifies the time-code as belonging to a certain type of event. A key benefit of this technique is that only one ‘Event Marker’ is required to create a highlight clip, where traditionally an ‘in’ and ‘out’ marker would need to be identified, specifying the start and end point of the clip. In particular, because the single “event marker” is used in conjunction with a ratio to identify the duration of the clip to be provided before as well as after the marked point, the user does not need to identify the beginning of the build-up to the event itself. A clip can be identified and generated easily and in real-time on a live video stream by a non-expert user through a single interaction at the moment of the key event.
A common approach to delivering video over HTTP (hypertext transport protocol) involves the dissection of large video files into multiple smaller files (called chunks), with each chunk corresponding to a segment of video perhaps a few seconds in length, and each with its own URL (universal resource locator). In such systems the a server platform provides a manifest file to the client, which specifies the URL of each chunk of the clip requested, so that the client can request, compile and then play back the video.
This invention utilises this delivery approach but modifies the way the manifest file is created and interpreted by the client device, so as to prioritise delivery of content based on its narrative importance.
For a ‘sports highlights’ show, the key events in a video sequence (such as goals) would be downloaded first, followed by further ‘video chunks’ in the sequence preceding and following each individual event. This enables ‘highlight clips’ to be provided in a flexible manner suited to the bandwidth available. This would be beneficial to viewers using low-capacity networks or those with unreliable coverage, for example a mobile user. This invention would allow the number of individual highlight clips made available to the viewer to be optimised in the presence of restrictions in bandwidth availability or connection reliability.
The prioritisation of video chunks would also enable video clips to be dynamically compiled ‘on the fly’ matched to specific play-back length restrictions. This facility may be useful when providing ‘later-corner’ catch-up facilities which would enable the viewer to be provided with a synopsis of key events which took place prior to joining the live stream.